feat(document-search): Implement document search public interface #58

akonarski-ds · 2024-10-03T10:03:17Z

Closes #56

github-actions · 2024-10-03T10:07:27Z

Trivy scanning results.

.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: JWT (jwt-token)
════════════════════════════════════════
JWT token
────────────────────────────────────────
.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA:80
────────────────────────────────────────
78 >>> encoded = jwt.encode({"some": "payload"}, "secret", algorithm="HS256")
79 >>> print(encoded)
80 [ *********************************************************************************************************
81 >>> jwt.decode(encoded, "secret", algorithms=["HS256"])
────────────────────────────────────────

.venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: Slack (slack-web-hook)
════════════════════════════════════════
Slack Webhook
────────────────────────────────────────
.venv/lib/python3.10/site-packages/litellm/proxy/_types.py:1288
────────────────────────────────────────
1286 alert_to_webhook_url: Optional[Dict] = Field(
1287 None,
1288 [ description="Mapping of alert type to webhook url. e.g. alert_to_webhook_url: {'budget_alerts': '*****************************************************************************'}",
1289 )
────────────────────────────────────────

github-actions · 2024-10-03T10:12:51Z

🧪 Test coverage: 92.43%

Code Coverage Summary

Filename                                                                                                Stmts    Miss  Cover    Missing
----------------------------------------------------------------------------------------------------  -------  ------  -------  --------------------------------
packages/ragbits-core/src/ragbits/core/__init__.py                                                          0       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/__init__.py                                               0       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/base.py                                                   4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/__init__.py                                                     4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/base.py                                                        33       4  87.88%   33, 52, 88, 97
packages/ragbits-core/src/ragbits/core/llms/litellm.py                                                     25       4  84.00%   8-9, 54, 85
packages/ragbits-core/src/ragbits/core/llms/local.py                                                       24      10  58.33%   8-9, 42-47, 57, 70-71
packages/ragbits-core/src/ragbits/core/llms/types.py                                                        8       2  75.00%   24, 28
packages/ragbits-core/src/ragbits/core/llms/clients/__init__.py                                             4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/clients/base.py                                                23       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/clients/exceptions.py                                          14       6  57.14%   7-8, 17, 26-27, 36
packages/ragbits-core/src/ragbits/core/llms/clients/litellm.py                                             50      10  80.00%   10-11, 70, 108, 122-127
packages/ragbits-core/src/ragbits/core/llms/clients/local.py                                               37      12  67.57%   11-12, 62-70, 92-103
packages/ragbits-core/src/ragbits/core/prompt/__init__.py                                                   2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/base.py                                                      18       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/parsers.py                                                   34       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/prompt.py                                                   106       3  97.17%   103, 107, 147
packages/ragbits-core/tests/unit/__init__.py                                                                0       0  100.00%
packages/ragbits-core/tests/unit/llms/__init__.py                                                           0       0  100.00%
packages/ragbits-core/tests/unit/llms/test_litellm.py                                                      63       0  100.00%
packages/ragbits-core/tests/unit/prompts/__init__.py                                                        0       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_parsers.py                                                   65       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_prompt.py                                                    97       0  100.00%
packages/ragbits-dev-kit/src/ragbits/dev_kit/__init__.py                                                    0       0  100.00%
packages/ragbits-dev-kit/src/ragbits/dev_kit/prompt_lab/__init__.py                                         0       0  100.00%
packages/ragbits-dev-kit/src/ragbits/dev_kit/prompt_lab/discovery/__init__.py                               2       0  100.00%
packages/ragbits-dev-kit/src/ragbits/dev_kit/prompt_lab/discovery/prompt_discovery.py                      33       2  93.94%   56-57
packages/ragbits-dev-kit/tests/unit/discovery/__init__.py                                                   0       0  100.00%
packages/ragbits-dev-kit/tests/unit/discovery/prompt_classes_for_tests.py                                  30       0  100.00%
packages/ragbits-dev-kit/tests/unit/discovery/test_prompt_discovery.py                                     18       0  100.00%
packages/ragbits-dev-kit/tests/unit/discovery/ragbits_tests_pkg_with_prompts/__init__.py                    2       1  50.00%   3
packages/ragbits-dev-kit/tests/unit/discovery/ragbits_tests_pkg_with_prompts/prompts/__init__.py            3       2  33.33%   2-4
packages/ragbits-dev-kit/tests/unit/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt1.py       14       0  100.00%
packages/ragbits-dev-kit/tests/unit/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt2.py       14       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/__init__.py                                    2       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/_main.py                                      45       1  97.78%   94
packages/ragbits-document-search/src/ragbits/document_search/documents/__init__.py                          0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/documents/document.py                         55       3  94.55%   52, 95, 124
packages/ragbits-document-search/src/ragbits/document_search/documents/element.py                          30       1  96.67%   34
packages/ragbits-document-search/src/ragbits/document_search/documents/sources.py                          46       9  80.43%   13-14, 58, 87, 106, 109, 120-122
packages/ragbits-document-search/src/ragbits/document_search/ingestion/__init__.py                          0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py               20       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/__init__.py                0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/base.py                   14       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/dummy.py                  11       1  90.91%   29
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured.py           30       5  83.33%   83-94, 98
packages/ragbits-document-search/src/ragbits/document_search/retrieval/__init__.py                          0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/__init__.py               0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/base.py                   5       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/noop.py                   5       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/__init__.py                0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/base.py                    6       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/noop.py                    7       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/vector_store/__init__.py                       0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/vector_store/base.py                          12       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/vector_store/chromadb_store.py                75       3  96.00%   10-11, 133
packages/ragbits-document-search/src/ragbits/document_search/vector_store/in_memory.py                     18       0  100.00%
packages/ragbits-document-search/tests/__init__.py                                                          0       0  100.00%
packages/ragbits-document-search/tests/helpers.py                                                           3       0  100.00%
packages/ragbits-document-search/tests/integration/__init__.py                                              0       0  100.00%
packages/ragbits-document-search/tests/integration/test_unstructured.py                                    38      24  36.84%   22-29, 37-43, 51-57, 65-72
packages/ragbits-document-search/tests/unit/__init__.py                                                     0       0  100.00%
packages/ragbits-document-search/tests/unit/test_chromadb_store.py                                         88       1  98.86%   34
packages/ragbits-document-search/tests/unit/test_document_processor.py                                     17       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_search.py                                        41       0  100.00%
packages/ragbits-document-search/tests/unit/test_documents.py                                              13       0  100.00%
packages/ragbits-document-search/tests/unit/test_elements.py                                               15       0  100.00%
packages/ragbits-document-search/tests/unit/test_gcs_source.py                                             14       0  100.00%
packages/ragbits-document-search/tests/unit/test_providers.py                                              20       0  100.00%
packages/ragbits-document-search/tests/unit/test_simple_vector_store.py                                    16       0  100.00%
TOTAL                                                                                                    1373     104  92.43%

Diff against main

Filename                                                                                        Stmts    Miss  Cover
--------------------------------------------------------------------------------------------  -------  ------  --------
packages/ragbits-document-search/src/ragbits/document_search/_main.py                             +11      +1  -2.22%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py       -1      -1  +4.76%
packages/ragbits-document-search/tests/unit/test_document_processor.py                             +6       0  +100.00%
packages/ragbits-document-search/tests/unit/test_document_search.py                               +27       0  +100.00%
packages/ragbits-document-search/tests/unit/test_providers.py                                      -2       0  +100.00%
TOTAL                                                                                             +41       0  +0.24%

Results for commit: 37f63e4

Minimum allowed coverage is 60%

♻️ This comment has been updated with latest results

… ingesting document

mhordynski

lgtm

akonarski-ds added 8 commits October 3, 2024 12:20

Refactor document processor, add param for specficing processor while…

f66f2c2

… ingesting document

Add insert elements method

8f198c1

Add search_config param to search method

d27229c

Add tests for document search

5e3292a

Add more and cleanup some tests

ec46bf6

Fix integration tests, update docstrings

8b75cf2

Handle diffrent aiohttp versions

734e1ce

Fix test

37f63e4

akonarski-ds force-pushed the feature/implement_document_search_interface branch from fb77473 to 37f63e4 Compare October 3, 2024 10:20

akonarski-ds self-assigned this Oct 4, 2024

mhordynski approved these changes Oct 7, 2024

View reviewed changes

akonarski-ds merged commit abbcdb0 into main Oct 7, 2024
3 checks passed

mhordynski deleted the feature/implement_document_search_interface branch November 21, 2024 21:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(document-search): Implement document search public interface #58

feat(document-search): Implement document search public interface #58

akonarski-ds commented Oct 3, 2024

github-actions bot commented Oct 3, 2024 •

edited

Loading

github-actions bot commented Oct 3, 2024 •

edited

Loading

mhordynski left a comment

feat(document-search): Implement document search public interface #58

feat(document-search): Implement document search public interface #58

Conversation

akonarski-ds commented Oct 3, 2024

github-actions bot commented Oct 3, 2024 • edited Loading

.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)

.venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)

github-actions bot commented Oct 3, 2024 • edited Loading

Code Coverage Summary

Diff against main

mhordynski left a comment

Choose a reason for hiding this comment

github-actions bot commented Oct 3, 2024 •

edited

Loading

github-actions bot commented Oct 3, 2024 •

edited

Loading